"The highlighted tokens are primarily Japanese content words, including nouns, verbs, and adjectives, as well as compound words and grammatical particles that contribute to the core meaning of sentences. There is a focus on tokens that form key semantic units, such as objects, actions, and attributes, often at the end or middle of words, and sometimes include numerals or foreign loanwords. These tokens are essential for conveying the main informational content in Japanese text."
Score Type | Accuracy | Precision | Recall | F1 score | TPR | TNR | FPR | FNR |
---|---|---|---|---|---|---|---|---|
detection | 0.62 | 1.0 | 0.24 | 0.387 | 0.24 | 1.0 | 0.0 | 0.76 |
fuzz | 0.73 | 0.926 | 0.5 | 0.649 | 0.5 | 0.96 | 0.04 | 0.5 |